Papers with low-resource tasks
Robust Neural Machine Translation for Abugidas by Glyph Perturbation (2024.eacl-short)
Copied to clipboard
| Challenge: | Neural machine translation systems are vulnerable when trained on limited data. |
| Approach: | They propose to add noise to the training phase to increase robustness of NMT systems trained on limited data. |
| Outcome: | The proposed training strategy overcomes noise and improves robustness for low-resource tasks for abugida glyphs. |
PCBERT: Parent and Child BERT for Chinese Few-shot NER (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to improve model performance on few-shot or zero-shot datasets are not effective for Chinese few- shot NER. |
| Approach: | They propose a prompt-based Parent and Child BERT for Chinese few-shot NER to train an annotating model on high-resource datasets and then discover more implicit labels on low-resourced datasets. |
| Outcome: | The proposed model can be used on Weibo and other Chinese NER datasets and it is shown to be effective in few-shot learning. |
EvoPrompt: Evolving Prompts for Enhanced Zero-Shot Named Entity Recognition with Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) is a low-resource task that requires supervised learning, but practical scenarios lack annotated data. |
| Approach: | They propose an Evolving Prompts framework that guides the model to better address these issues through continuous prompt refinement. |
| Outcome: | The proposed framework shows consistent performance improvements on four benchmarks. |
XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for active learning rely on model uncertainty or disagreement to pick unlabeled data, leading to over-confidence in superficial patterns and lack of exploration. |
| Approach: | They propose to use a bi-directional encoder and a uni-directional decoder to generate and score an explanation for low-resource text classification. |
| Outcome: | The proposed model improves on 9 strong baselines on six datasets and can generate explanations for its predictions. |
Memorisation versus Generalisation in Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | State-of-the-art pre-trained language models have been shown to memorise facts and perform well with limited amounts of training data. |
| Approach: | They propose to extend pre-trained language models to generalise and memorise facts in noisy and low-resource scenarios. |
| Outcome: | The proposed extension improves performance in low-resource named entity recognition tasks. |
Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks. |
| Approach: | They propose a new compression paradigm that extracts knowledge from pre-trained language models to construct a knowledge store from which the model can leverage it for effective inference. |
| Outcome: | The proposed model extracts knowledge from LLMs to construct a knowledge store, which the model can leverage for effective inference. |
Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Sparsely gated Mixture of Experts (MoE) models are a compute-efficient method to scale model capacity for multilingual machine translation tasks. |
| Approach: | They propose a regularization strategy that prevents over-fitting of MoE models on low-resource tasks and conditional MoE Routing and curriculum learning methods that prevent over- fitting. |
| Outcome: | The proposed methods improve the performance of MoE models on low-resource tasks without adversely affecting high-res tasks. |